The present disclosure relates to a hand tracking system and method, for example, a hand tracking system and method for determining hand poses and gestures of an obstructed hand using an ultrasound sensor on a head-mounted display.
It is known that motion tracking systems can use image processing to determine the position and gestures of a person. Existing systems can use different types of cameras (for example, structured light scanners, RGB cameras, and depth cameras) to capture images and video of a person. However, the person must be in the field of view of the camera. For instance, if the person's right hand is obstructed by the person's left hand, then the camera cannot capture images or video of the right hand. Accordingly, image based motion tracking systems are unable to determine the position of obstructed body parts.
An ultrasound sensor can determine the distance between the ultrasound sensor and another object or surface. The ultrasound sensor emits an ultrasound signal, i.e., a sound wave between 20 kiloHertz and 200 megaHertz. The ultrasound signal reflects off of the object or surface, and the ultrasound sensor detects the reflected ultrasound signal. The ultrasound sensor determines the distance based on the known speed of sound (approximately 340 meters per second) and the time of flight of the reflected ultrasound signal.